Adversarial training (AT) is a powerful regularization method for neuralnetworks, aiming to achieve robustness to input perturbations. Yet, thespecific effects of the robustness obtained by AT are still unclear in thecontext of natural language processing. In this paper, we propose and analyze aneural POS tagging model that exploits adversarial training (AT). In ourexperiments on the Penn Treebank WSJ corpus and the Universal Dependencies (UD)dataset (28 languages), we find that AT not only improves the overall taggingaccuracy, but also 1) largely prevents overfitting in low resource languagesand 2) boosts tagging accuracy for rare / unseen words. The proposed POS taggerachieves state-of-the-art performance on nearly all of the languages in UDv1.2. We also demonstrate that 3) the improved tagging performance by ATcontributes to the downstream task of dependency parsing, and that 4) AT helpsthe model to learn cleaner word and internal representations. These positiveresults motivate further use of AT for natural language tasks.
展开▼
